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Abstract 

Designing experiential learning activities requires an instructor to think about what they want 
the students to learn. Using importance-performance analysis can assist with the 
instructional design of the activities. This exploratory study used importance-performance 
analysis in an online introduction to criminology course. There is limited research on experiential 
learning in online courses as well as empirical data to assist with the instructional design of the 
experiential learning activities. The primary goal of this article is to demonstrate the use of 
importance- perfonnance analysis to guide the instructional design of experiential learning 
activities. 
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Introduction 

Experiential learning is a philosophy that draws on the work from prominent 20 th century 
scholars such as John Dewey, Kurt Lewin, Jean Piaget and others. This is a learning experience 
or process that allows students to apply classroom concepts to real-world situations (Kolb & 
Kolb, 2005). Experiential learning can take many forms including service learning, wilderness- 
based adventure programs, and professional development activities (Carver, 1997). Traditional 
experiential learning typically follows Kolb’s experiential learning theory (Kolb & Lewis, 1986). 
This theory provides a holistic model for the learning process and a multilinear model of adult 
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development (Kolb, Boyatzis, & Mainemelis, 2001). Kolb’s experiential learning theory is based 
on a four-stage learning cycle: concrete experience, reflective observation, abstract 
conceptualization, and active experimentation. Effective learning has been observed when 
learners progress through the cycle. 

The benefits of experiential learning have been well established (Waldner, McGory, & 
Widener, 2010), as research has shown that students in experiential learning courses had more 
positive course evaluations (Markus, Howard, & King, 1993). Other research supports positive 
beliefs and values toward service and community (Cohen & Kinsey, 1994), higher academic 
achievement (Boss, 1994), and positive impact on personal, attitudinal, moral, social, and 
cognitive outcomes (Giles & Eyler, 1994). 

Higher education enrollment is increasing according to the Institute of Education 
Statistics (IES). There are approximately 17 million students enrolled in higher education 
institutions in the United States; that number is predicted to jump to over 20 million by 2021. 
Online enrollment has increased from 1.2 million students in 2002 to 7.1 million students in 
2012 (Allen & Seaman, 2014). Online course offerings are also growing to accommodate a 
diverse student audience. According to the National Center for Educational Statistics from 2000- 
2012, distance education program enrollment went from 2% in 2000 to 16.3% in 2012 (The 
Condition of Education, 2014). 

As more courses and programs offer online options, it also becomes important to 
integrate and research experiential learning in online courses. Waldner, McGorry, and Widner 
(2012) noted very limited research with regards to online experiential learning. The few 
empirical studies that have been conducted on online experiential learning have identified 
common challenges and best practices but have conducted case study or descriptive research 
without reporting results. In addition, many of these studies failed to compare what is important 
to students to how they perceived their perfonnance. 

Literature Review 

Kolb’s Experiential Learning Theory 

John Dewey developed a philosophy of education that describes the process of infusing 
experiences into the learning environment. This philosophy has become the foundation for 
experiential learning theory. Through an extensive review of Dewey’s philosophy, Giles and 
Eyler (1994) identified nine components to consider for experiential learning: continuity of 
experience, citizenship, interaction, inquiry, reflection, educative products, concrete and abstract 
knowledge, the great community, and democracy. By implementing experiential learning theory, 
students experience both process and outcomes (Carver, 1997). Students are able to apply 
knowledge and skills gained in the classroom through real-world experience. One of the key 
factors in Dewey’s philosophy is creating individual learning experiences through reflection 
(Deans, 1999). 

Kolb further developed the experiential learning theory based on Dewey’s philosophy 
and Piaget’s psychology for experiential learning implementation. Kolb’s experiential learning 
theory includes four learning activities: concrete experience, reflective observation, abstract 
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conceptualization, and active experimentation (Petkus, 2000; Kolb & Lewis, 1986). Experiential 
learning theory is a holistic model of learning processes and a model for adult development. 

The experiential learning theory is based on a learning cycle where two activities are for 
grasping experience: concrete experience and abstraction conceptualization. The other two 
activities are for transfonning that experience: reflective observation and active experimentation. 

Through experiential learning, the learner experiences all four activities. Concrete experience is 
the basis for reflective observation (Kolb & Kolb, 2009). These reflections are conformed into 
abstract conceptualizations from which students may draw new conclusions. These conclusions 
are then integrated into active experimentation. Together, these activities when integrated 
together create a positive experiential learning experience for students, community partners, and 
faculty (Kolb & Kolb, 2009). 

Experiential learning involves teaching and learning strategies that can include 
community service or activities, hands on experience, and critical reflection (Waldner et ah, 
2010) to course concepts (McGorry, 2012). Research has supported the benefits of experiential 
learning. Eyler, Giles, Stenson, and Gray, (2001) identified 135 articles on experiential learning 
where 132 articles reported positive or neutral consequences. Faculty benefits included 
satisfaction in student learning and research; student benefits included critical analysis, 
application of knowledge in practical settings, and improved course satisfaction (Eyler et al., 
2001 ). 

Online Experiential Learning 

Online experiential learning is experiential learning that is incorporated into courses 
delivered in an online fonnat. Malvey, Hamby, and Fottler (2006) defines it as 

... an electronic form of experiential education and [it] incorporates 
electronically supported experiential learning. It is delivered online and uses the 
Internet and state of the art technologies that pennit students, faculty, and 
community partners to collaborate at a distance in an organized, focused, 
experiential service learning activity, which simultaneously promotes civic 
responsibility and meets community needs (p. 187). 

Waldner et al. (2012) conducted a thorough review of experiential learning literature in 
an online enviromnent using a three-pronged approach. The approach searched for literature 
using the peer review database ProQuest Central, an Internet search using Google and Google 
Scholar, and a search of journals dedicated to experiential learning. Only 18 journal articles and 
one book were found to be directly related to online experiential learning. The result of this 
extensive literature review established that there is little empirical research related to online 
experiential learning. 

The studies in online experiential learning yielded the following results regarding best 
practices: 

• Technology 

• Communication 

• Course design 
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Waldner et al. (2012) stated that specific technology should be identified to facilitate 
communication between the instructor, students, and potential community partner(s), and clear 
communication expectations should be established prior to course delivery and be included in the 
course design for students. Along with technology and communication expectations, course 
design should be taken into account since the integration of online experiential learning can 
increase instructor workload (Waldner et ah, 2012). All parties including outside agencies, 
instructors and students, must be trained on the communication technology and have the 
technology tested prior to implementation. The instructor should partner with instructional 
designers for course design and technology support (Waldner et al., 2010). Course design should 
be “solid”. Instructors should evaluate courses according to course design rubrics that have been 
established and validated to ensure there is a connection between the activities and the course 
goals (Waldner et al., 2012). The course should include clear instructions and explanations of 
reflective assignments, gather student input on the course and activities, and ensure 
communication expectations are included within the course design (Waldner et al., 2010; 
Waldner et al., 2012). In both cases, the students highly valued solving authentic real-world 
problems as noted on end-of-course evaluations. 

McGorry (2012) conducted a case study research with two online marketing classes and 
two traditional marketing classes. This was a comparison study to detennine if online 
experiential learning delivered the same benefits as traditional experiential learning based on 
self-reported perceptions of experiential learning outcomes using the Service Learning Benefit 
scale (SELEB). SELEB measures student perceptions of experiential learning activities. From 
this study, the author concluded there was no significant difference between the online group and 
the traditional group with regards to student perceptions but further studies should explore 
student grades and perceptions (McGorry, 2012). 

Student Perceptions and IPA 

Student perceptions often have been used to gauge perceived achievement instead of 
measuring actual achievement (Kams, 2005). Students frequently are asked questions regarding 
whether the experiential activity deepened their learning, yet they typically are not asked what 
they feel is important prior to the experiential learning activity. Markus et al. (1993) researched 
student perceptions towards experiential learning and then compared the mean grade between the 
control class and the class that integrated experiential learning. Overall the mean grades in the 
experiential learning class were significantly higher. 

Importance-performance analysis (IPA) has been applied in higher education in various 
contexts. It has typically been utilized in education for marketing and tourism courses (Oh, 
2001). IPA has also been used for student evaluation of teaching and course design (Huybers, 
2014; Lewis, 2004) 

The importance survey is designed to be implemented at the beginning of the semester 
and could also be considered a pre-survey. The performance survey is deployed at the end of the 
semester is considered a post-survey. For the purposes of this paper, the surveys will be 
described in tenns of importance and perfonnance (pre-/post-). The SELEB importance- 
performance survey consist of four factors: practical skills, interpersonal skills, citizenship, and 
personal responsibility. Twenty variables are associated with the four factors as seen in Table 1 
(Toncar, Reid, Burns, Anderson, & Nguyen, 2006). The central distribution for importance and 
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performance are calculated (means or median) and the score is used to calculate the axis for a 
two-dimensional matrix called an action grid (Siniscalchi, Beale, & Fortuna, 2008). Figure 1 
depicts what Ortinau, Bush, Bush, and Twible (1989) have defined for each quadrant: Data in the 
upper right quadrant (high importance/high perfonnance) is “keep up the good work”; data 
landing in the upper and lower left quadrants (low importance/high perfonnance and low 
importance/low performance) suggests overkill or low priority. Data landing in the lower right 
quadrant (high importance/low performance) indicates importance outweighs the ability. 
Importance-perfonnance surveys have been used to guide changes in service industries but in 
this case, importance-perfonnance is use to analyze the outcomes of the design of the 
experiential learning activity. 
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Figure 1. Importance-performance analysis (IPA) 


After a review of the literature, there is a clear gap on effectively integrating online 
experiential learning into online asynchronous courses and whether students perceive online 
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experiential learning to be beneficial to their learning based on the importance of the skills 
learned. Also note that none of the online experiential studies discussed the actual results of 
surveys delivered nor included the surveys in the appendix. 

Methods 


Research Strategy 

This quasi-experimental study explored students’ perceptions regarding online 
experiential learning in terms of what they perceived to be important and how they performed. 

The study utilized an importance-performance analysis (IPA) method. 

Participants 

This study used an Introduction to Criminology course with majors and non-majors, 
which was delivered in fall 2014 during a fifteen-week semester. This is an undergraduate course 
at a mid-sized southeastern university in the United States. The asynchronous criminology 
course had 29 students. Students were recruited to participate in this study on a voluntary basis at 
the beginning of the semester. Students who chose to participate in the study, were assigned a 
random identification number which allowed the researcher to anonymously match survey data 
with grades for statistical analysis. Fifteen students completed both the importance and 
perfonnance survey with a 51.7% response rate. This response rate falls in line with the 
recommended response rate for online surveys (Baruch & Holtom, 2008). Student demographics 
showed that 14 students age 18-30 and one student 31-50. There were 7 males and 8 females. 
There were a variety of student classifications including 2 freshmen, 3 sophomores, 6 juniors, 
and 4 seniors. Twelve students stated they were comfortable using computers and 3 who 
identified themselves as advanced. A majority of the students had taken online classes with 8 
having taken 5-10 courses. Two students had taken 1-2, four had taken 3-4, and one had taken 
more than 10 online courses. 

Materials and Procedure 

The SELEB survey was used to gauge student perceptions of the experiential learning 
and questions relating to demographic data were added to the electronic survey. The SELEB 
scale is a self-report measure of student perceptions of the benefits of service learning. 
According to Toncar et al. (2006), the SELEB survey has been identified as a valid and reliable 
tool for assessing outcomes of student perceptions, value of experiential learning efforts from the 
student viewpoint, and evaluate the extent to which the experiential learning activity contributed 
to the learning objectives of the course. The Cronbach’s alpha of the SELEB scale was 
calculated at .82 (McGorry, 2012). Toncar, et al.’s evaluation (2006) suggested the SELEB scale 
should be used to assess the benefits of experiential learning. 

The importance SELEB scale includes a 12-item Likert scale on the importance of 
incorporating various items such as civic duty, connecting theory to practice, and workplace 
skills in course activities. This is given to the students at the beginning of the semester. The scale 
is 1 - 7 with 1 being not important and 7 being very important. The perfonnance SELEB scale is 
a Likert 20-item scale with 1 indicating the project was not applicable and 7 being very 
applicable. The 20-item perfonnance SELEB is deployed to the students at the end of the 
semester once the experiential learning activity had concluded. Table 1 displays the survey items 
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and how each are associated with each main factor. There are four factors associated with the 
twenty variables. 


Table 1 

Factors and Associated Variables of the SELEB Scale 


Practical Skills 

Interpersonal Skills 

Citizenship 

Personal 

Responsibility 

Applying Knowledge 
to the “Real World” 

Person growth 

Understanding 
cultural and racial 
differences 

Caring relationships 

Problem Analysis and 

Ability to work well 

Social responsibility 

Being trusted by 

Critical Thinking 

with others 

and citizenship skills 

others 

Social Self- 

Leadership Skills 

Community 

Empathy and 

Confidence 


involvement 

sensitivity to the 
plight of others 

Conflict Resolution 

Workplace skills 

Skills in Learning 
from Experience 
Organizational Skills 
Connecting Theory 
with Practice 

Communication skills 

Ability to make a 
difference in the 
community 



The instructor included a summary of the study on Blackboard ©, the Learning 
Management System (LMS) at the university, for the students. The 12-Likert scale importance 
SELEB survey and questions pertaining to demographic infonnation was given electronically to 
the students at the beginning of the semester. The survey included class status, age, sex, major, 
previous experience with online learning, and classification of “on-campus” or “distance 
education” student. At the end of the semester, the 24-Likert questions perfonnance SELEB was 
deployed electronically to the participants that gauged student perceptions towards the criminal 
justice in action activity, which was the experiential learning activity that spanned the semester, 
at the end of the semester. 

Criminal Justice in Action 

For the experiential learning project, students were asked to identify a location in their 
community to visit and observe. The students were required to conduct two one-hour-long 
observations. Prior to community observations, students wrote an essay which included 
expectations and intentions based on the class materials. Students also completed the 
importance-SELEB scale to assess their perceptions towards experiential “community-based” 
learning. Once students completed their observations, they were paired with another student in 
the class to create a final presentation of their findings and explain how the observations related 
to the class materials. The students created a blog, a letter to the editor, or a voice-over 
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PowerPoint presentation. Finally, students were asked to compose a critical reflection essay on 
their experience and complete the performance-SELEB survey to assess their perceptions 
towards the “community-based” experiential learning. 

Data Analysis 

IPA was used to analyze student perceptions and perfonnance. This data was used to 
construct a two-dimensional matrix where importance is depicted on the y-axis and perfonnance 
is depicted on the x-axis. The gridlines for the IPA graphs are detennined by the overall mean of 
Importance = 6.02) and Perfonnance ( = 4.85). 

A paired-samples t-test was conducted to evaluate the impact of the experiential learning 
assignment using importance-perfonnance scale. There was a significant difference between 
importance and performance for all associated variables except for the Applying Knowledge to 
the Real World. There was a statistically significant decrease in the perfonnance scores. Table 2 
displays the mean scores for the four main factors of the SELEB survey. There was a decrease in 
the mean from importance to perfonnance. 

Table 2 

Mean comparison of the SELEB factors 


Factors 

Importance 

Performance 

Practical Skills 

6.03 

4.92 

Interpersonal Skills 

6.02 

4.69 

Citizenship 

6.04 

4.85 

Personal Responsibility 

5.96 

4.41 


A paired-samples t-test was conducted to analyze the four main factors comparing the 
importance to the perfonnance. Practical skills resulted in a decrease from importance (M = 
6.033, SD = .949) to performance (M= 4.748, SD = 1.858), t (14) = -3.335 ,p = .005 (two-tailed). 
The mean decrease in the practical skills scores was -1.293 with a 95% confidence interval 
ranging from .458 to 2.112. The eta squared statistic (.44) indicated a moderate effect size. 
Interpersonal skills resulted in a decrease from importance (M = 5.952, SD = 1.579) to 
perfonnance (M= 4.446, SD = 1.544), t (13) = -4.415,/? = .001 (two-tailed). The mean decrease 
in the interpersonal scores was -1.506 with a 95% confidence interval ranging from .769 to 
2.242. The eta squared statistic (.60) indicated a moderate effect size. Citizenship skills resulted 
in a decrease from importance (M= 6.044, SD = .871) to performance (M= 4.60, SD = 1.929), t 
(14) = -3.3511, p = .003 (two-tailed). The mean decrease in the citizenship scores was -1.444 
with a 95% confidence interval ranging from .562 to 2.326. The eta squared statistic (.47) 
indicated a moderate effect size. Personal responsibility skills resulted in a decrease from 
importance (M= 5.892, SD = .1.583) to perfonnance (M= 4.446, SD = 1.544), t (13) = -4.206,/? 
= .001 (two-tailed). The mean decrease in the personal responsibility scores was -1.446 with a 
95% confidence interval ranging from .703 to 2.189. The eta squared statistic (.58) indicated a 
moderate effect size. 
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Table 3 

t-test importance-performance of SELEB factors 


Factor 

t 

Sig. (2-tailed) 

Eta squared 

Practical Skills 

-3.33 

.005 

0.44 

Interpersonal Skills 

-4.41 

.001 

0.60 

Citizenship 

-3.51 

.003 

0.47 

Personal 

-4.20 

.001 

0.58 

Responsibility 





Table 4 depicts the analysis of the two-tailed t-test for the variables of the four main factors. 
Each item is labeled which is depicted in Figure 2 on the IP A quadrant. There is a decrease in the 
mean scores from importance to performance. 


Table 4 

t-ttest importance-performance of SELEB items. Letters correspond with Figure 2 



SELEB Items 

Mean 

Importance 

Mean 

Perfonnance 

t 

Sig. (2- 
tailed) 

Eta squared 

A 

Applying 
knowledge to the 
"Real World" 

6.53 

5.47 

-1.92 

.076 

0.21 

B 

Workplace skills 

6.07 

4.33 

-3.17 

.007 

0.42 

C 

Organizational 

skills 

5.87 

4.47 

-2.50 

.025 

0.31 

D 

Understanding 
cultural and racial 
difference 

5.87 

4.47 

-2.17 

.048 

0.25 

E 

Social 

responsibility and 
citizenship skills 

5.80 

4.80 

-3.24 

.006 

0.43 

F 

Social self- 
confidence 

5.67 

4.20 

-2.95 

.010 

0.38 

G 

Ability to assume 

personal 

responsibility 

6.00 

4.40 

-2.82 

.014 

0.36 

H 

Gaining the trust 
of others 

5.93 

4.00 

-3.65 

.003 

0.49 

I 

Ability to work 
with others 

5.93 

4.14 

-3.70 

.003 

0.51 

J 

Leadership skills 

6.07 

4.53 

-3.29 

.005 

0.44 

K 

Communication 

skills 

6.00 

4.60 

-2.28 

.004 

0.45 


Note. *p < .05, two-tailed 
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Figure 2 illustrates the associated variables from the importance and performance SELEB 
scale on an importance-performance quadrant. The x and v axis is determined by determining the 
overall mean score for both importance and performance (Huybers, 2014). The overall 
importance mean was 6.022 and overall performance was 4.854. Applying real world skills 
landed in the upper right quadrant, which suggests that this activity met both the importance and 
perfonnance for students. Workplace and leadership skills fall between the upper left 
(concentrate here) and lower left (low priority) quadrants. All other skills fell in the low priority 
quadrant. 



2 2.5 3 3.5 4 4.5 5 5.5 6 

Performance 


Figure 2. Results from the importance-performance mean. 

Discussion and Conclusion 

The results in this experiential learning activity demonstrated that students were able to 
apply classroom knowledge to real world activities. Although there was a decrease in the mean 
for all associated variables of the SELEB survey, this was an introductory class that included 
both majors and non-majors. The instructor’s main objective in this introductory course was to 
connect theory with practice; thus the experiential learning activity was successful. Many of the 
studies conducted in online experiential learning thus far have neither included the results of 
their surveys nor used an instrument designed for experiential/service learning. The results of 
this exploratory study can assist with the design of experiential learning activities depending on 
the goals of the instructor and the activity/course. Since this course was an introductory course 
that included majors and non-majors, students may not have found value in the other associated 
variables. The goal of the instructor was for students to connect classroom concepts to the real 
world and this was successful. 
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Limitations and Future Research 

The use of importance-performance analysis can be an important evaluation tool to guide 
the design of experiential learning whether in online or face-to-face environments. This study 
had several limitations. One limitation was the sample size. This study applied to only one 
introductory class and students had the option to participate, which likely reduced the number of 
participants. Sample size could be increased by using multiple sections of the same course. 
Another limitation was the fact that the course was an introductory course that included both 
majors and non-majors; therefore, students may not have perceived the experiential learning 
activity to be beneficial. Perhaps future studies could compare majors and non-majors to 
detennine if perceptions change. Further studies should be conducted across disciplines to make 
the findings more generalizable. 
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APPENDIX A 
The 20-Item SELEB Scale 

Please indicate how well your dass project has provided you with the following educational experience with 1 being not at all 
and 7 being very much so. 


Personal Growth 

7 

6 

5 

4 

3 

2 

1 

Ability to Work with Others 

7 

6 

5 

4 

3 

2 

1 

Leadership Skills 

7 

6 

5 

A 

3 

2 

1 

Communication Skills 

7 

6 

5 

A 

3 

2 

1 

Understanding Cultural and Racial Differences 

7 

6 

5 

A 

3 

2 

1 

Social Responsibility and Citizenship Skills 

7 

6 

5 

A 

3 

2 

1 

Community Involvement 

7 

6 

5 

A 

3 

2 

1 

Applying Knowledge to the "Real World" 

7 

6 

5 

A 

3 

2 

1 

Problem Analysis and Critical Blinking 

7 

6 

5 

A 

3 

2 

1 

Social Self-Conlidence 

7 

6 

5 

A 

3 

2 

1 

Conflict Resolution 

7 

6 

5 

A 

3 

2 

1 

Ability to Assume Personal Responsibility 

7 

6 

5 

A 

3 

2 

1 

Development ot Caring Relationships 

7 

6 

5 

A 

3 

2 

1 

Gaining the Trust of Others 

7 

6 

5 

A 

3 

2 

1 

hmpathy and Sensitivity to the Plight of Others 

7 

6 

5 

A 

3 

2 

1 

Workplace Skits 

7 

6 

5 

A 

3 

2 

1 

Ability to Make a Difference in the Community 

7 

6 

5 

A 

3 

2 

1 

Skills in Learning from txperience 

7 

6 

5 

A 

3 

2 

1 

Organization Skills 

7 

6 

5 

A 

3 

2 

1 

Connecting Theory with Practice 

7 

6 

5 

A 

3 

2 

1 


Notes: The instructions above were used to assess the benefits of service learning among the two classes in the study. When developing the scale, the 
Instructions were: "Please Indicate how Important each of the following are to you In your educational experience with one being not at all and seven 
being very much so." 


APPENDIX B 
The SELEB Scale 


Please indicate how important each of the following are to you in your educational experience with 1 being not at all and 7 being 
very much so. 


Applying Knowledge to the "Real World" 

1 2 

3 

4 

5 

6 

7 

Workplace Skills 

1 2 

3 

4 

5 

6 

7 

Organization Skills 

1 2 

3 

4 

5 

6 

7 

Understanding Cultural and Racial Differences 

1 2 

3 

4 

5 

6 

7 

Social Responsibility and Gtizenship Skills 

1 2 

3 

4 

5 

6 

7 

Ability to Make a Difference in the Community 

1 2 

3 

4 

5 

6 

7 

Social Self Confidence 

1 2 

3 

4 

5 

6 

7 

Ability to Assume Personal Responsibility 

1 2 

3 

4 

5 

6 

7 

Gaining the trust of Others 

1 2 

3 

4 

5 

6 

7 

Ability to Work with Others 

1 2 

3 

4 

5 

6 

7 

1 eadership Skills 

1 2 

3 

4 

5 

6 

7 

Communication Skills 

1 2 

3 

4 

5 

6 

7 
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